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EXECUTIVE SUMMARY 


States and districts have expressed interest in administering the 
ACT® to 10th-grade students. Given that the ACT was designed to 
be administered in the spring of 11th grade or fall of 12th grade, the 
appropriateness of this use should be evaluated. As such, the focus 
of this paper is to summarize empirical evidence evaluating the 

use of the ACT as a measure of college readiness for 10th graders. 
In alignment with a Standards for Educational and Psychological 
Testing (AERA, APA, & NCME, 2014), empirical evidence related 

to five sources of validity evidence (response processes, internal 
structure, content, relation to other variables, and consequences) are 
summarized. As compared to 11th-grade test administrations, the 
results indicate that when the ACT is administered to all 10th graders: 


¢ Students are similarly motivated (response process) 

¢ Scores are only slightly less reliable (internal structure) 

« ACT scores and test completion rates are predictably lower 
for 10th graders relative to 11th graders 


Additionally, the results indicate that: 


¢« The content of the test is aligned to college readiness 
standards based on what students learn in high school and 
need to know to succeed in college (content) 

¢ How students perform on the ACT in 10th grade is 
comparable to other 10th-grade measures (i.e., PreACT®) of 
college readiness 

« ACT scores from 10th grade are predictive of high school 
grades and 11th grade ACT scores (relation to other 
variables) 


Unlike other 10th grade tests, the ACT provides a college-reportable 
score and greater test security (relative to the PreACT test). Relative 
to 11th-grade ACT testing, the primary drawback of 10th-grade ACT 
testing is that students have less time for academic development and 
are less likely to have taken courses that can contribute to higher 
ACT scores. While most uses of 10th-grade ACT test scores are also 
possible with other assessments, the primary benefit of ACT testing 
is that it provides a secure measure for admissions to postsecondary 
programs. Collectively, the evidence supports the use of the ACT 

as a measure of college readiness or academic preparation level for 
10th-grade students. 
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Introduction 


Since 1959, ACT® scores have been used for 
many purposes, including college admissions, 
placement, scholarship eligibility, and talent 
identification. In some states, the ACT is also 
used as a measure of school accountability — 
either as an academic achievement indicator, a 
college readiness indicator, or both. Because 
the ACT is often associated with high-stakes 
decisions for students, those students are 
motivated to perform their best. In addition, other 
members of educational communities (e.g., 
parents, school leaders, community leaders, and 
policymakers) have a vested interest in how 
their students perform on the test. 


The ACT is administered on national test dates, 
testing primarily college-bound students, and 
through state and district testing programs, 
testing all students. In both cases, students are 
usually in the spring of 11th grade or fall of 12th 
grade when they take the test. There is also 
some interest among state and district leaders in 
spring grade 10 testing. This interest has 
multiple reasons: 


e To give all students an early opportunity to 
place into college (dual enrollment) courses 


e To provide authentic practice for students to 
take the ACT again in grade 11 


e To obtain information from a secure and 
reliable test that students take seriously 


e To determine if students are on target for 
college and career readiness 


e To help with evaluation of school/district 
programs and curricula 


However, given that the ACT is typically given in 
11th and 12th grade, there may be concerns 
that ACT scores for a 10th-grade testing 
program would not be appropriate measures of 
academic achievement. Further, assessments 
such as the PreACT™, ACT Aspire®, and other 
state assessments can fulfill some of these 10th- 
grade testing needs. Thus, there is a need to 
evaluate, for different assessment goals and 
contexts, if and when the ACT test is an 
appropriate measure for 10th-grade populations 
and to examine the merits of the ACT relative to 
other 10th-grade assessments. 


Evidence from Early 
Adopters of 10th-grade 
ACT Testing 


Some schools and districts across the United 
States have already adopted the ACT test for 
both 10th and 11th graders. We have analyzed 
data from these early adopters to better 
understand differences in the validity and 
reliability of ACT scores for 10th grade relative 
to 11th grade. 


In the spring of the 2016-2017 or 2017-2018 
academic years, 53 schools from 15 states 
administered the ACT to virtually all of their 
10th- and 11th-grade students through state or 
district testing programs. Of the 53 schools, 19 
are located in rural locales, nine in towns, eight 
in suburbs, and 17 in urban locales; 34 were 
public schools and 19 were non-public. 


The schools tested 5,415 10th-grade students 
and 5,712 11th-grade students.’ The sample was 
52% female, 48% male, 65% White, 20% African 
American, 8% Hispanic, 2% Asian, and 5% other 
race/ethnicity. Approximately one-third (33%) 
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were low-income (<$36K), 36% had parental 
income from $36K-$80K, and 31% had family 
income greater than $80K. The 10th- and 11th- 
grade samples were very similar in terms of the 
demographic distribution by sex, race/ethnicity, 
and parents’ income. The average 11th-grade 
ACT Composite score was 19.8, which is slightly 
higher than the average across all schools that 
participated in 11th-grade state or district testing 
programs in spring 2018 (19.0). We use this 
sample of schools and students to make 
comparisons of 10th- and 11th-grade ACT 
administrations on: 


e Course-taking rates for courses most related 
to higher ACT scores 


e College reporting (score-sending and 
enrollment in ACT’s Educational Opportunity 
Service) 


e Test completion and guessing patterns 
e Score reliability 


e Test performance (score averages and 
college readiness rates) 


Course-Taking 


End-of-course assessments typically measure 
content knowledge tied to a specific high school 
course. Conversely, the ACT measures 
achievement with respect to college and career 
readiness in general and requires examinees to 
integrate knowledge, skills, and abilities (KSAs) 
from a variety of courses. In particular, ACT 
administers the National Curriculum Survey® 
(ACT, 2016) every three to five years, and the 
results of the survey inform test design and 
content. Teachers respond to a series of 
questions regarding the importance and coverage 
of specific skills for courses in elementary school, 
middle school, high school, and college. 
Because the ACT is designed to measure the 
most essential content, skills, and concepts 
taught in high school and needed for success in 
college and career, content evidence exists 
supporting the use of the ACT to measure high 
school students’ academic preparation level. 


Research shows that students who take 
challenging courses during high school and 
perform well in those courses tend to achieve 
higher ACT scores (McNeish, Radunzel, & 
Sanchez, 2015). In particular, students tend to 
perform significantly better after taking upper- 
level math and science coursework. One reason 
for this is that the ACT tests not only concepts 
that are introduced in Algebra 1 and Geometry, 
but also concepts taught in Algebra 2 courses. 
And, the science test draws content for its 
passages from Biology, Chemistry, and Physics. 
While the science test does not focus on course- 
specific content knowledge, there are several 
questions that do require this knowledge, and 
ACT’s research shows that students who have 
taken more science courses tend to perform 
better on the test. Students’ ACT scores also 
increase with additional English and social 
studies coursework. However, upper-level 
courses in these areas are less likely to 
introduce new and specific concepts that are 
tested by the ACT. 


Testing later in high school (e.g., spring of 11% 
grade or fall of 12th grade) helps students 
achieve higher ACT scores because they have 
more time for coursework and more time to 
develop the skills needed to integrate knowledge 
from multiple courses and apply it to different 
contexts. Table 1 compares math and science 
course-taking rates for the 10th- and 11th-grade 
student groups. The vast majority of students in 
both groups took Algebra 1, but 11th graders 
were more likely to have taken Geometry (95% 
vs. 85%), Algebra 2 (88% vs. 52%), Biology 
(98% vs. 88%), Chemistry (78% vs. 53%), and 
Physics (30% vs. 13%). As expected, the 
differences are largest for courses that are 
commonly taken in 11th grade (Algebra 2 and 
Chemistry). The differences in course-taking 
suggest that 10th graders are more likely to 
encounter some unfamiliar ACT test content 
than are 11th graders. 
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Table 1 also compares the correlation of high 
school GPA and ACT Composite score for the 
two groups. The correlation is slightly higher for 
11th graders (0.55 vs. 0.51). The correlations 
confirm the expectation that students who 
perform better in high school courses tend to 
perform better on the ACT test, and this is true 
for 10th- and 11th-grade ACT administrations. 
The correlation may be slightly higher for 11th 
graders due to having taken more courses and 
thus having a more reliable measure of high 
school GPA. 


College-Reporting 


As a way to increase access (consequential 
validity evidence), students can send their ACT 
scores to up to four colleges at no additional 
cost.2 In addition, students can opt into ACT’s 
Educational Opportunity Service (EOS) to 
increase their exposure to, and knowledge of, 
postsecondary opportunities. One might expect 
these college reporting behaviors to be different 
for the 10th- and 11th-grade groups. For 
example, 10th-grade students might be more 
hesitant to send scores, knowing that they will 
likely perform better on the ACT when they’re in 
11th grade. Students who are not motivated to 
perform their best on the ACT might also be less 
likely to report their scores. However, we found 
that college reporting behaviors are very similar 
for the two groups. The majority of 10th graders 
(66%) and 11th graders (70%) sent scores to at 
least one college, and differences in EOS opt-in 
rates were small (61% for 10th graders, 66% for 
11th graders). This suggests that 10th graders 
are only slightly less likely to engage in college 
reporting, and these data do not suggest lower 
motivation for 10th graders. 


Test Completion and Guessing 
Patterns 


Some students are not able to complete the 
ACT test in the allotted time. Others resort to 
guessing to complete as much of the test as 
possible, which is what ACT advises since there 
is no penalty for guessing. Incomplete tests or 
guessing patterns could be due to difficulty 


working through the test items, low motivation, 
or other factors. Such indicators provide 
evidence in relation to response processes. If 
test incompletion rates or guessing indicators 
were markedly higher for the 10th-grade group, 
it could signal that the ACT test is too difficult for 
10th graders or that 10th grade students are not 
as motivated. 


Test Completion 


We found that test completion rates were high 
for both groups, ranging from 84% for English 
grade 10 to 94% for science grade 11 (Table 1). 
The largest difference between the two groups 
was for English (89% for 11th grade, 84% for 
10th grade). The data show that 11th graders 
are slightly more likely to complete the test, 
which may be due to their additional 
coursework, higher academic achievement, 
increased test familiarity, and/or higher 
motivation. As a contrast, we also compared 
completion rates for the 10th-grade PreACT test 
and 11th-grade ACT test (Table 2). The 
PreACT/ACT comparisons in Table 2 are based 
on a sample of 55,450 students who took the 
PreACT in spring grade 10 and the ACT in 
spring grade 11, thus is a comparison of the 
same students. Test completion rates are very 
similar for the 10th-grade PreACT and 11th- 
grade ACT, with the largest difference observed 
for reading (93% for PreACT, 91% for ACT). 
Note that the PreACT tests are shorter than their 
ACT counterparts, which could explain the 
smaller difference in completion rates between 
10th-grade PreACT and 11th-grade ACT as 
compared to 10th-grade ACT and 11th-grade 
ACT. Because 10th-grade test completion rates 
are lower than 11th-grade completion rates 
when the ACT is used as the 10th-grade test (as 
compared to the PreACT), we can conclude that 
taking the ACT in 10th grade results ina 
speeded experience for more students as 
compared to taking the PreACT in 10th grade, 
but the effect is small. 
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Guessing Rates 


To estimate the extent to which guessing 
occurs, we computed the percentage of 
examinees with rapid guessing response 
patterns.? Across subjects, guessing patterns 
ranged from 10% (science grade 11) to 23% 
(English grade 10; Table 1). The percentage of 
examinees with rapid guessing response 
patterns was not appreciably higher for 10th 
graders as compared to 11th graders, with the 
exception of the English results (23% for 10th as 
compared to 18% for 11th graders). Because 
the test completion and guessing rates are 
similar for the two groups, the evidence 
suggests similar levels of motivation for 10th and 
11th graders. 


Score Reliability 


As evidence of internal structure, reliability of ACT 
scale scores are estimated for both 10th and 11th 
graders in Table 1.4 The reliability of the ACT test 
scores were only slightly lower for the 10th-grade 
sample as compared to the 11th-grade sample 
with the largest observed difference for science 
(.77 for 10th grade, .81 for 11th grade). The 
reliability estimates for the ACT Composite score 
were .95 and .96 for 10th graders and 11th 
graders, respectively. These results suggest that 
the ACT is a reliable measure of college readiness 
for 10th-grade students. 


Test Performance 


Relative to 10th-grade students, 11th-grade 
students are expected to perform better on the 
ACT test due to having additional time for 
coursework and development of the skills needed 
to integrate knowledge across different learning 
experiences. However, because the ACT is not 
designed to assess the specific curricula of any 
given course or set of courses (and instead is 
based on curricula across the national landscape 


of courses), we might not expect large 
differences between the two groups. We found 
that the 11th-grade group scored higher, with an 
average Composite score of 19.8 compared to 
18.5 for the 10th-grade group. This difference is 
comparable to the average one-year Composite 
score gain from the 10th-grade PreACT to the 
11th-grade ACT of 1.7 (Table 2). Across subject 
tests, the differences ranged from 1.0 for Math to 
1.8 in English. 


The percentage of students meeting the ACT 
College Readiness Benchmarks is an indicator 
of the percentage of students who would likely 
be considered ready for a college-level high 
school course (e.g., dual enrollment). As 
expected, the percentages are lower for 10th 
graders relative to 11th graders. The difference 
is largest for English (57% for 11th grade, 47% 
for 10th grade). For the other subjects, the 11th- 
grade group is higher by 8-9 percentage points. 
As a contrast to the grade 10 ACT/grade 11 
ACT comparison, we also examined grade 10 
PreACT/grade 11 ACT college readiness 
differences using the 55,450 students who took 
the PreACT in spring grade 10 and the ACT test 
in spring grade 11 (Table 2). If the two sets of 
differences are similar, it suggests that students 
are equally likely to demonstrate readiness on 
10th-grade ACT and 10th-grade PreACT 
administrations. Across subjects, the grade 11 
ACT readiness rates were higher than the grade 
10 PreACT readiness rates by 10-12 percentage 
points, very similar to the first set of differences. 
Collectively, these results suggest that a 
considerable percentage of 10th graders are 
ready for college courses, with room to improve 
as 11th graders. The results also suggest that if 
a state decides to use the ACT to assess 10th 
graders’ academic achievement levels and 
college readiness, it would obtain similar results 
to what would be obtained if it chose to use the 
PreACT for the same purpose. 
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Table 1. Same-School Comparison of 10th- and 11th-Grade ACT Spring Administrations 


Grade Level 
10 11 
Measures (N=5,415) (Ete at 4) Difference 
Algebra 1 98% 99% 1% 
Geometry 85% 95% 10% 
Gaurses Taken Algebra 2 52% 88% 37% 
Biology 88% 98% 10% 
Chemistry 53% 78% 25% 
Physics 13% 30% 17% 
Ghiese Reaaring Score sending 66% 70% 4% 
EOS participation 61% 66% 5% 
English 84% 89% 5% 
Test Completion ve ll ote “e 
Reading 89% 90% 1% 
Science 92% 94% 1% 
English 23% 18% -5% 
Guessing Patterns Main Lei we Ee 
Reading 14% 14% 0% 
Science 11% 10% -1% 
English 0.89 0.92 0.02 
Math 0.83 0.87 0.04 
Score Reliability Reading 0.84 0.86 0.02 
Science 0.77 0.81 0.05 
Composite 0.95 0.96 0.01 
English 17.9 19.7 1.8 
Math 18.1 19.1 1.0 
Score Averages Reading 18.6 20.0 1.4 
Science 18.7 19.7 1.0 
Composite 18.5 19.8 1.3 
English 47% 57% 11% 
College Readiness liad he oe ea 
Reading 28% 37% 9% 
Science 20% 28% 8% 


Correlation with HSGPA Composite 0.51 0.55 0.04 
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Table 2. Same-Student Comparison of 10th-Grade PreACT and 11th-Grade ACT Spring Administrations® 


Test / Grade Level 


Measures PreACT 10 ACT 11 Difference 
English 90% 89% -1% 
Test Math 86% 86% 0% 
completion Reading 93% 91% -2% 
Science 92% 93% 0% 
English 18.6 211 25 
ans Math 19.4 20.9 AS 
Reading 20.2 21:7 1.5 
averages 
Science 19.9 21.3 1.4 
Composite 19.7 21.4 1:7 
English 55% 66% 11% 
College Math 32% 43% 11% 
readiness Reading 38% 48% 10% 
Science 29% 41% 12% 


Table 3. PreACT and ACT Reliability Estimates® 


PreACT 
STU] 0) [2Xe Grade 10 Difference 
English 0.88 0.92 0.04 
Math 0.88 0.91 0.03 
Reading 0.83 0.87 0.04 
Science 0.82 0.85 0.03 
Composite 0.96 0.97 0.01 


Benefits of 10th-grade ACT 
Testing, Relative to Other 
Assessments 


Tenth grade administration of the ACT test could 
fulfill multiple needs, including college 
placement, practice for the ACT test, 
accountability reporting, and measuring student 
growth. Next, we discuss the benefits of using 
the ACT for these purposes, relative to other 
10th grade assessments. 


College placement. Tenth grade ACT testing 
could give all students, including underserved 
learners who may not have considered college 
as an option, an early opportunity to be placed in 
college courses (e.g., dual enrollment). Because 
it's a secure test, an established measure of 


college readiness, and is designed to measure 
skills important for success in college (ACT, 
2016, 2017), the ACT test may be preferred over 
other tests for this purpose. Scores from other 
tests would likely be predictive of success in 
college courses, though less predictive than 
ACT test scores. 


Practice. Tenth grade ACT testing provides the 
most authentic practice for the 11th-grade ACT 
test. The PreACT test also provides an ACT 
practice experience, but the test length and 
other specifications are different. On the other 
hand, the PreACT may provide a more engaging 
practice experience than the ACT test, given 
that it is designed with 10th graders in mind. To 
a lesser extent, ACT Aspire Interim and 
Summative Assessments can also serve as a 
practice experience for the ACT test, as they 
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test the same subject areas and domains (and and the PreACT also report predicted ACT score 
have the same reporting categories, plus more) ranges. Tenth grade PreACT and ACT test 

as both the PreACT and ACT tests. All three scores are equally predictive of grade 11 ACT 
tests identify students who are on target for test scores, with ACT Aspire scores nearly as 
readiness for first-year college courses, predictive (Table 4). All three tests can also be 
providing ample time for students to skill up used to measure student growth for purposes of 
while still in high school. Free resources are also research and evaluation. Because PreACT and 
available to help students prepare for the ACT ACT scores are reported on the same scale, 
test. ACT Academy™ is an online tool that growth from grade 10 to grade 11 is easier to 
includes two full-length ACT practice tests, conceptualize when the PreACT or ACT test is 
quizzes, and a diagnostic and recommendation used in 10th grade. However, student growth 
system that directs students to online resources percentiles can be produced for all three 

tailored to their needs (https://academy.act.org/). assessments and do not require a common 
Outside of a high-stakes standardized testing scale. Currently, grade 10 to grade 11 ACT 
event, students could gain valuable practice and student growth percentiles are available for ACT 
further develop skills important for college Aspire and PreACT (ACT, 2019). 

success through ACT Academy. While the 

testing experience is not the same as an actual Table 4. Correlations With 11th-Grade ACT Test 
ACT test administration, engaged students are Scores 


likely to benefit from the additional support 
offered through ACT Academy. 


10th Grade Test 


rXeoay 

Accountability. The ACT Aspire Summative Subject Aspire’ PreACT?® ACT? 
Assessments and the ANT tet are both used to English 0.82 0.85 0.85 
meet state and federal testing requirements for 

accountability. Both tests require secure test Math 0.82 0.86 0.86 
administrations and produce reliable measures Reading 0.70 0.78 0.78 
of academic achievement. Tenth grade students Science 0.75 0.76 0.76 
may be more motivated to perform their best on Composite 0.88 0.91 0.91 
the ACT test, given its use ior admissions, ELA 0.83 _ 0.89 
course placement, scholarships, and talent 

recognition. On the other hand, ACT Aspire and STEM 0.85 0.87 0.88 


state assessments may be more appropriate 
tests for 10th-grade populations, given that they 
are designed to test KSAs for that grade level. 
Any test used for federal accountability needs to 
meet requirements for alignment to state 
standards. ACT Aspire has been used as an 
accountability test for grades 3-10, and the ACT 
test has been used for 11th grade. 


While 10th-grade administration of the ACT 
offers some benefits over ACT Aspire, PreACT, 
and other state assessments, cost is another 
important consideration for testing programs. 
The ACT test is generally more expensive than 
ACT Aspire Summative, and ACT Aspire 
Summative is generally more expensive than the 
PreACT and ACT Aspire Interim. 


Growth. ACT Aspire, PreACT, and the ACT test 
can all be administered in 10th grade to gauge 
whether students are on target for meeting 
college and career readiness benchmarks in 
11th grade. Both ACT Aspire and PreACT have 
readiness benchmarks, indicating whether 
students are on target to meet the ACT College 
Readiness Benchmarks in grade 11. ACT Aspire 
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Concl usions Relative to 11th-grade ACT administrations, the 
primary drawback of 10th-grade ACT test 

In sum, the evidence | administrations is that students are less likely to 

summarized in this document s / have taken courses linked to higher ACT scores 


supports the use of the ACT ~™ 
as a measure of student’s 

college readiness or 

academic preparation level os 
for 10th-grade students. 


— and would have less time for academic 
development. While most uses of 10th-grade 
ACT test scores are also possible with other 

BS assessments, the primary benefit of ACT testing 
is that it provides a secure measure for 
admissions to postsecondary programs. These 


Pi 


In building a validity argument for this use case, results can help inform state and district choices 
we didn’t rely on a single source of evidence but for a 10th-grade assessment. 

rather took a comprehensive approach, 

documenting evidence as it relates to all five 

sources of validity evidence. 


Notes 


1. 


Of the 53 schools, seven administered the ACT in 2016-2017 only, 43 administered the ACT in 2017-2018 only, 
and three administered the ACT both years. For schools that administered the ACT both years, there are 
students who took the ACT in both 10th grade (2016-2017) and 11th grade (2017-2018). However, we compare 
all 10th-grade administrations to all 11th-grade administrations. 

Students with an ACT Fee Waiver can send their ACT scores to up to six colleges (at the time of registration) 
and an additional 20 regular score reports after registration at no additional cost. 

Rapid guessing patterns include long strings of item responses using the same response option, various 
repetitive patterns, or consecutive blanks. 

The reliability analyses were conducted only for one test form which had sufficiently large sample sizes (>1000) 
for both grades 10 and 11. These statistics were estimated with a four-parameter beta compound binomial 
model, taking into account the content specifications (Kolen, Hanson, & Brennan, 1992). The reliability of the 
composite scores was calculated based on the SEMs of the English, mathematics, reading, and science test 
scores, assuming that the measurement errors of these tests are independent (ACT, 2016). 

Statistics based on a sample of 55,450 students who took the ACT in spring grade 11 in 2018 and the PreACT in 
spring grade 10 in 2017. 

Reliability estimates are obtained from the ACT technical manual (ACT, 2017) and from estimates obtained from 
PreACT tests administered during the 2016-2017 academic year. 

Correlation estimates based on students who took the ACT in 11th grade (from 2014 through 2018) and ACT 
Aspire in 10th grade (from 2013 through 2017). 

Correlation estimates based on a sample of 55,450 students who took the ACT in spring grade 11 in 2018 and 
the PreACT in spring grade 10 in 2017. 

Correlation estimates based on a sample of students who took the ACT in spring grade 11 as part of a state or 
district testing program in 2018 and also took the ACT in spring grade 10 in 2017. N=2,771 for ELA; N=34,429 


for the other subject areas. 
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